Controlled Markov chains with safety upper bound

Controlled Markov chains with safety upper bound

In this paper we introduce and study the notion of safety control of stochastic discrete event systems (DESs), modeled as controlled Markov chains. For non-stochastic DESs, modeled by state machines or automata, safety is specified as a set of forbidden states, or equivalently by a binary valued vector that imposes an upper bound on the set of states permitted to be visited. We generalize this ...

Controlled Markov Chains and Safety Criteria

On Controlled Markov Chains with Optimality Requirement and Safety Constraint

We study the control of completely observed Markov chains subject to generalized safety bounds and optimality requirement. Originally, the safety bounds were specified as unit-interval valued vector pairs (lower and upper bounds for each component of the state probability distribution). In this paper, we generalize the constraint to be any linear convex set for the distribution to stay in, and ...

Approximations for Markov Chains with Upper Hessenberg Transition Matrices

Abstract We present an approximation for the stationary distribution T of a countably infinite-state Markov chain with transition probability matrix P = (pq) of upper Hessenberg form. Our approximation makes use of an associated upper Hessenberg matrix which is spatially homogeneous one P ( ~ ) except for a finite number of rows obtained by letting p q = pj-i+l, i > N + 1, for some distribution...

Stationary Probabilities of Markov Chains with Upper Hessenberg Transition Matrices

In this paper, based on probabilistic arguments, we obtain an explicit solution of the stationary distribution for a discrete time Markov chain with an upper Hessenberg time stationary transition probability matrix. Our solution then leads to a numerically stable and efficient algorithm for computing stationary probabilities. Two other expressions for the stationary distribution are also derive...